System with half-depth servers

ABSTRACT

A computer system includes a chassis, one or more hard disk drives coupled to the chassis, and one or more air passages under at least one of the hard disk drives. The air passages include one or more air inlets and one or more air outlets. The inlets direct at least a portion of the air downwardly into the passages. The passages allow air to move from the air inlets to the air outlets.

This application is a continuation of U.S. application Ser. No.13/791,192, filed Mar. 8, 2013, now U.S. Pat. No. 9,122,462, which is acontinuation of U.S. application Ser. No. 12/886,472, filed Sep. 20,2010, now U.S. Pat. No. 8,400,765, which are hereby incorporated byreference in their entireties.

BACKGROUND

Organizations such as on-line retailers, Internet service providers,search providers, financial institutions, universities, and othercomputing-intensive organizations often conduct computer operations fromlarge scale computing facilities. Such computing facilities house andaccommodate a large amount of server, network, and computer equipment toprocess, store, and exchange data as needed to carry out anorganization's operations. Typically, a computer room of a computingfacility includes many server racks. Each server rack, in turn, includesmany servers and associated computer equipment.

Computer systems typically include a number of components that generatewaste heat. Such components include printed circuit boards, mass storagedevices, power supplies, and processors. For example, some computerswith multiple processors may generate 250 watts of waste heat. Someknown computer systems include a plurality of such larger,multiple-processor computers that are configured into rack-mountedcomponents, and then are subsequently positioned within a rack system.Some known rack systems include 40 such rack-mounted components and suchrack systems will therefore generate as much as 10 kilowatts of wasteheat. Moreover, some known data centers include a plurality of such racksystems.

In some computer systems, a rack-level power distribution unit isprovided in a rack to distribute electrical power to the many servers inthe rack. The rack-level power distribution unit may include a largenumber of receptacles, each of which may be used to supply power to adifferent server. Rack-level power distribution units may be attached toone or both interior sides of the rack near one end of the rack. Thismounting may place the receptacles of the power distribution unit inclose proximity to the server power supplies. Such a mounting may,however, interfere with installation and removal of the server racks(for example, by obstructing the path of a server as it is slid into orout of the rack).

In one existing arrangement for a rack-based computer system, coolingair is introduced at the front end of a rack and into the front ofservers mounted in the rack. Heated air is expelled through the rear ofthe server chassis and then exits through the rear of the rack. In manysystems, electrical connections for power and data are at also locatedthe rear of the rack system, along with associated equipment for theelectrical connections such as rack-level power distribution units. Withthe electrical connections located at hot end of the rack, personnel maybe forced to work in a hot environment to maintain the servers (forexample, to connect and disconnect power and data cables). In addition,high temperatures at the rear of the rack may cause failures in therack-level power distribution units (for example, due to thermaloverload of breakers in the power distribution units).

A source of heat in many servers comes from on-board power supply unitsin the servers. Power supply units that are not properly cooled may besusceptible to failure. Many standard power supply units include aninternal fan that draws air from inside the server chassis into thepower supply case and then expels heated air to a location external tothe server through a face panel in the power supply unit. Thisarrangement may be effective in cooling electrical components in thepower supply unit. In some cases, however, the heat air expelled fromthe power supply unit may adversely affect cooling of other componentsin the server or in other parts of the system. For example, air expelledfrom a power supply unit may pre-heat air being introduced into a serverchassis to cool critical components in the server, such as a centralprocessing unit.

Some servers include a substantial number of hard disk drives (forexample, eight or more hard disk drives) to provide adequate datastorage. The hard disk drives include motors and electronic componentsthat generate heat, which must be removed from the hard disk drives tomaintain operation of the servers. This heat is sometimes removed bypassing air over and on the sides the cases of the hard disk drives. Inmany existing servers, the hard disk drives are laterally spaced fromone another to allow air to pass between adjacent hard disk drives tocool of the hard disk drives. Such lateral spacing may be needed toensure adequate cooling of the hard disk drives, but also may have theeffect of limiting the maximum density of the hard drives (that is, thenumber of hard disk drives that can be provided in a given amount ofspace in the server.)

Some data centers rely on DC-powered fans internal to servers to produceairflow through the servers in a rack system. Such fans may be, however,inefficient and prone to failure, and add to the costs and complexity ofthe servers. DC fans also require power conversion equipment (eitherinternal or external to the servers) to supply the DC power for thefans.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates one embodiment of a system that includes electricalconnections and intake vents at the front end of a rack system.

FIG. 2 illustrates one embodiment of a computer system that may bemounted in a rack.

FIG. 3 illustrates one embodiment of a data center having power and dataconnections on a cold-aisle side of a row of racks.

FIG. 4 illustrates air flow through one embodiment of a computer system.

FIG. 5 illustrates a rear view of a computer system including a harddisk drive tray.

FIG. 6 is a schematic side view illustrating air flow through a computersystem according to one embodiment.

FIG. 7 is a schematic top view illustrating air flow through a computersystem according to one embodiment.

FIG. 8 illustrates one embodiment of a rack power distribution unit on acoupling device that allows for movement of the rack power distributionunit.

FIG. 9 is a top view schematic illustrating one embodiment of a racksystem with rack power distribution units in a normal position.

FIG. 10 is a top view schematic illustrating one embodiment of a racksystem with rack power distribution units in a maintenance position.

FIG. 11 illustrates a schematic top view illustrating one embodiment ofa data center having a row of server cold-cold rack systems with rackpower distribution units on both ends of the racks.

FIG. 12 illustrates a schematic end view illustrating one embodiment ofa data center having a row of server cold-cold rack systems with rackpower distribution units on both ends of the racks.

FIG. 13 is a top view schematic illustrating a power connection for arack system with a rack power distribution unit in a closed position.

FIG. 14 is a top view schematic illustrating a power connection for arack system with a rack power distribution unit in an open position.

FIG. 15 illustrates an embodiment of a rack system with two powerdistribution units on a single bracket.

FIG. 16 is a schematic view illustrating an embodiment of a rack systemincluding a rack power distribution unit coupled to a rack by way of alinkage.

FIG. 17 is a schematic view illustrating an embodiment of a rack systemincluding a rack power distribution unit in a sliding arrangement.

FIG. 18 illustrates an embodiment of a rack system that includes frontrack power distribution units and rack doors.

FIG. 19 illustrates one embodiment of a tray for hard disk drives in acomputer system.

FIG. 20 illustrates one embodiment of a clamping bar in a raisedposition from the base of a tray.

FIG. 21 illustrates a side view of one embodiment of a tray for harddisk drives.

FIG. 22 illustrates a cross section of locking bar on adjacent diskdrives.

FIG. 23 illustrates a tray with clamping bars in an open position toallow removal of hard disk drives from the tray.

FIG. 24 illustrates one embodiment a locking mechanism for hard diskdrive tray.

FIG. 25 illustrates a rear view of one embodiment of a rack system.

FIG. 26 illustrates one embodiment of a fan module for a rack.

FIG. 27 illustrates one embodiment of a door for supporting fans in arack system.

FIG. 28 illustrates a rear view of a portion of a fan module.

FIG. 29 illustrates a side view of a fan module.

FIG. 30 illustrates one embodiment of a fan with an adjustable mounting.

FIG. 31 illustrates an adjustable fan adjusted to a vertical angle.

FIG. 32 illustrates a method of cooling components in a rack-mountedcomputer system according to one embodiment.

FIG. 33 illustrates one embodiment of a reconfiguration or maintenanceoperation that includes moving rack power distribution units to accesscomputer systems in a rack.

FIG. 34 illustrates a method of cooling hard disk drives by flowing airunder the drives according to one embodiment.

FIG. 35 illustrates a method of cooling computer systems usingrack-mounted fans according to one embodiment.

While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and will herein be described in detail. Itshould be understood, however, that the drawings and detaileddescription thereto are not intended to limit the invention to theparticular form disclosed, but on the contrary, the intention is tocover all modifications, equivalents and alternatives falling within thespirit and scope of the present invention as defined by the appendedclaims. The headings used herein are for organizational purposes onlyand are not meant to be used to limit the scope of the description orthe claims. As used throughout this application, the word “may” is usedin a permissive sense (i.e., meaning having the potential to), ratherthan the mandatory sense (i.e., meaning must). Similarly, the words“include,” “including,” and “includes” mean including, but not limitedto.

DETAILED DESCRIPTION OF EMBODIMENTS

Various embodiments of computer systems, and systems and methods ofcooling computer systems, are disclosed. According to one embodiment, adata center includes a row of one or more racks. Computer systems (suchas servers) are mounted in the racks. A cold aisle is on a first side ofthe rack row and a hot aisle is on the second side of the rack row. Anair handling system moves air from the cold aisle on the first side ofthe row of racks through computer systems in at least one of the racksand exhausts air from the computer systems into the hot aisle on thesecond side of the row of racks. The computer systems includeinput/output connectors, power input connectors, and power supply airinlets on the first side (cold-aisle side) of the row. One or more powerrack power distribution units may be provided on the first side(cold-aisle side) of the row.

According to one embodiment, a system includes a rack and one morecomputer systems (such as servers) mounted in the rack. The computersystems may include an air intake side including air inlets, an airexhaust side including air outlets, and a power supply unit. One or morefans move air through at least one of the air inlets of the air intakeside, across one or more heat producing components of the computersystem, and through at least one of the air outlets of the air exhaustside, and a power supply unit. The power supply unit includes a powersupply enclosure enclosing at least one of the heat producingcomponents, a power supply module in the enclosure, and one or morepower supply air inlets. The power supply module supplies power toelectrical components external to the enclosure. The power supply airinlets allow air into the power supply enclosure. At least one of thepower supply air inlets is located on the air intake side of thecomputer system.

According to one embodiment, a rack-mountable computer system includes achassis that is mountable in a rack, and a power supply unit coupled tothe chassis. The power supply unit includes a power supply enclosure, apower supply module, and a fan. The power supply enclosure has a panelthat faces outwardly from the chassis. The outwardly facing panelincludes one or more openings for air. The fan pulls air through theopenings in the panel and across one or more heat producing componentsof the power supply module.

According to one embodiment, a method of cooling components includesproviding an air moving device for heat producing components of arack-mounted computer system. Air is drawn from outside the rack-mountedcomputer system into a power supply enclosure for the power supply unit.The air is expelled from the power supply enclosure into an enclosurefor the rack-mounted computer system. In certain embodiments, a standardpower supply unit is modified to reverse a direction of air flow in thepower supply unit.

According to one embodiment, a system includes a rack and one or morecomputer systems mounted in the rack. One or more rack powerdistribution units are coupled in the rack one or more sides of therack. The rack power distribution units supply power to the computersystems in the rack. The rack power distribution units may be rotatablewith respect to the rack to allow installation or removal of thecomputer systems on the side of the rack on which the rack powerdistribution units are mounted.

According to one embodiment, a system includes a rack, one or morecomputer systems mounted in the rack, one or more rack powerdistribution units, and one or more coupling devices. The couplingdevices couple the rack power distribution units to the rack such thatthe rack power distribution unit is movable relative to the rack whilethe rack power distribution units remains installed on the rack to allowinstallation or removal of the computer systems on the side of the rackon which the rack power distribution units are mounted.

According to one embodiment, a coupling device for coupling a rack powerdistribution unit to a rack includes one or more brackets and one ormore rack coupling portions. The brackets include one or more PDUcoupling portions that couple with a rack power distribution unit. Therack coupling portions allow movement of the bracket relative to therack.

According to one embodiment, a method of performing maintenance orreconfiguration on computer systems in a rack includes moving a rackpower distribution unit from an operating position to a maintenanceposition while the rack power distribution unit remains coupled to therack. When the rack power distribution unit is in the maintenanceposition, the rack power distribution unit is out of theinstallation/removal path for computer systems in the rack. A computersystem is at least partially removed from or installed in the rack whilethe rack power distribution unit is in the maintenance position.

According to one embodiment, a computer system includes a chassis, oneor more hard disk drives coupled to the chassis, and one or more airpassages under at least one of the hard disk drives. The air passagesinclude one or more air inlets and one or more air outlets. The inletsdirect at least a portion of the air downwardly into the passages. Thepassages allow air to move from the air inlets to the air outlets.

According to one embodiment, a tray for holding one or more data storagedevices includes one or more support portions that support the datastorage devices and one or more spacing portions that establish one ormore air passages below the data storage devices when the tray isinstalled in a computer system.

According to one embodiment, a method includes moving air into passagesunder hard disk drives of a computer system, allowing heat from heatproducing components on the hard disk drives to transfer to air in thepassages, and removing the air from the passages.

According to one embodiment, a system includes a rack, one or morecomputer systems mounted in the rack, and two or more alternatingcurrent (AC) fans mounted in the rack. The AC fans move air through thecomputer systems mounted in the rack. At least one of the AC fans canmove air through at least two of the computer systems mounted in therack

According to one embodiment, a system includes a rack, one or morecomputer systems mounted in the rack, and one or more fans coupled tothe rack at an angular orientation. The fans move air through thecomputer systems mounted in the rack. At least one of the fans can moveair through at least two of the computer systems mounted in the rack.

According to one embodiment, a system for providing air flow throughcomputer systems in a rack includes a mounting panel that mounts in therack and one or more fan modules. The fan modules include a fan chassisand one or more fans. The mounting panel holds the fan modules at anangle with respect to the rack.

According to one embodiment, a method includes coupling AC fans to arack such that the AC fans are in an angular orientation relative to therack, and operating the AC fans to move air through computer systems inthe rack.

As used herein, “air intake side” is a side of a system or element, suchas a server, that can receive air into the system or element.

As used herein, “air exhaust side” is a side of a system or element,such as a server, that can exhaust, expel, or discharge air from thesystem or element.

As used herein, “air handling system” means a system that provides ormoves air to, or removes air from, one or more systems or components.

As used herein, “air moving device” includes any device, element,system, or combination thereof that can move air. Examples of air movingdevices include fans, blowers, and compressed air systems.

As used herein, an “aisle” means a space next to one or more racks.

As used herein, “ambient” means, with respect to a system or facility,the air surrounding at least a portion of the system or facility. Forexample, with respect to a data center, ambient air may be air outsidethe data center, for example, at or near an intake hood of an airhandling system for the data center.

As used herein, a “cable” includes any cable, conduit, or line thatcarries one or more conductors and that is flexible over at least aportion of its length. A cable may include a connector portion, such asa plug, at one or more of its ends.

As used herein, “chassis” means a structure or element that supportsanother element or to which other elements can be mounted. A chassis mayhave any shape or construction, including a frame, a sheet, a plate, abox, a channel, or a combination thereof. A chassis for a computersystem may support circuit board assemblies, power supply units, datastorage devices, fans, cables, and other components of the computersystem.

As used herein, “computing” includes any operations that can beperformed by a computer, such as computation, data storage, dataretrieval, or communications.

As used herein, “computer system” includes any of various computersystems or components thereof. One example of a computer system is arack-mounted server. As used herein, the term computer is not limited tojust those integrated circuits referred to in the art as a computer, butbroadly refers to a processor, a server, a microcontroller, amicrocomputer, a programmable logic controller (PLC), an applicationspecific integrated circuit, and other programmable circuits, and theseterms are used interchangeably herein. In the various embodiments,memory may include, but is not limited to, a computer-readable medium,such as a random access memory (RAM). Alternatively, a compact disc-readonly memory (CD-ROM), a magneto-optical disk (MOD), and/or a digitalversatile disc (DVD) may also be used. Also, additional input channelsmay include computer peripherals associated with an operator interfacesuch as a mouse and a keyboard. Alternatively, other computerperipherals may also be used that may include, for example, a scanner.Furthermore, in the some embodiments, additional output channels mayinclude an operator interface monitor and/or a printer.

As used herein, “coupling device” includes an element or combination ofelements that can be used to couple one element or structure to one ormore other elements or structures. Examples of a coupling device includea bracket, a linkage, a connecting rod, a hinge, a rail, or combinationthereof.

As used herein, “data center” includes any facility or portion of afacility in which computer operations are carried out. A data center mayinclude servers dedicated to specific functions or serving multiplefunctions. Examples of computer operations include informationprocessing, communications, testing, simulations, power distribution andcontrol, and operational control.

As used herein, “data center module” means a module that includes, or issuitable for housing and/or physically supporting, one or more computersystems that can provide computing resources for a data center.

As used herein, to “direct” air includes directing or channeling air,such as to a region or point in space. In various embodiments, airmovement for directing air may be induced by creating a high pressureregion, a low pressure region, or a combination both. For example, airmay be directed downwardly within a chassis by creating a low pressureregion at the bottom of the chassis. In some embodiments, air isdirected using vanes, panels, plates, baffles, pipes or other structuralelements.

As used herein, a “duct” includes any device, apparatus, element, orportion thereof, that can direct, segregate, or channel a fluid, such asair. Examples of ducts include cloth or fabric ducts, sheet metal ducts,molded ducts, tubes, or pipes. The cross sectional shape of a passagewayof a duct may be square, rectangular, round or irregular, and may beuniform or change over the length of the duct. A duct may be aseparately produced component or integral with one or more othercomponents, such as a frame.

As used herein, a “module” is a component or a combination of componentsphysically coupled to one another. A module may include functionalelements and systems, such as computer systems, circuit boards, racks,blowers, ducts, and power distribution units, as well as structuralelements, such a base, frame, housing, or container.

As used herein, a “pin” includes any element that can be positioned toconstrain or hold another element in a desired position or orientation.Suitable pins may include straight pins, pegs, threaded bolts,unthreaded bolts, bars, plates, hooks, rods, or screws.

As used herein, “power distribution unit” means any device, module,component, or combination thereof, that can be used to distributeelectrical power. The elements of a power distribution unit may beembodied within a single component or assembly (such as a transformerand a rack power distribution unit housed in a common enclosure), or maybe distributed among two or more components or assemblies (such as atransformer and a rack power distribution unit each housed in separateenclosure, and associated cables, etc.).

As used herein, a “rack” means a rack, container, frame, or otherelement or combination of elements that can contain or physicallysupport one or more computer systems.

As used herein, “rack power distribution unit” refers to a powerdistribution unit that can be used to distribute electrical power tovarious components in a rack. A rack power distribution unit may includevarious components and elements, including wiring, bus bars, connectors,and circuit breakers.

As used herein, “room” means a room or a space of a building. As usedherein, “computer room” means a room of a building in which computersystems, such as rack-mounted servers, are operated.

As used herein, a “space” means a space, area or volume.

In some embodiments, electrical connections and intake air forrack-mounted computer systems are provided at a common end of the racksystem. FIG. 1 illustrates one embodiment of a system that includeselectrical connections and intake vents at the front end of a racksystem. System 100 includes rack 102 and computer systems 104. Computersystems 104 are installed on front posts 106 and rear posts 108 of rack102.

Each of computer systems 104 includes chassis 112, circuit boardassembly 114, power supply unit 116, and hard disk drives 118. In someembodiments, circuit board assembly 114 is a motherboard for computersystem 104. In certain embodiments, a power supply unit may supply powerto two or more motherboard assemblies in a computer system. In someembodiments, each of computer systems 104 is 1 rack unit (1 U) high. Inone embodiment, rack 102 is a 42 U rack.

At front 119 of computer system 104, computer system 104 includesinput/output panel 120 and air vents 122. Power supply unit 116 includespower supply panel 132. Power supply panel 132 includes input powerreceptacle 134 and air vents 136.

Rack 102 includes base 140, posts 142, and top panel 144. Posts 142include front hinge elements 146 and rear hinge elements 148. In someembodiments, rack 102 may include panels on any or all of the sides,front, and rear of the rack 102 (a right side panel of rack 102 has beenpartially omitted in FIG. 1 for clarity).

Rack 102 includes rear door 160. Rear door 160 is coupled to rack 102 onrear hinge elements 148. Rear door 160 includes fans 162. Fans 162 maybe operated to provide air flow through computer systems 104. In oneembodiment, fans 162 create a low pressure region inside rack 102 at therear of computer systems 104. Air may be drawn from the front of rack102 through computer systems 104 and exhausted out the rear of rack 102through fans 162.

Rack 102 includes rack power distribution units 170. Rack powerdistribution units 170 are mounted on PDU brackets 172. PDU brackets 172include hinges 174. PDU brackets 172 are coupled to rack posts 142 atfront hinge elements 146. Rack power distribution units 170 include PDUoutput receptacles 176. Rack power distribution units 170 may supplypower to computer systems 104. For each computer system 104, one ofpower cables 178 may couple one of output receptacles 176 in rack powerdistribution unit 170 with input receptacle 134 on power supply unit 116of the computer system.

Power connectors for a power cable and the corresponding receptacles ona rack power distribution unit and power supply may be any of variousconnector types. In one embodiment, rack power distribution units 170have IEC C13 receptacles and power supply units 116 have IEC C14receptacles.

Rack power distribution unit 170 may have any suitable powercharacteristics. Examples of output voltages for rack power distributionunitl 170 include 100 volts, 110 volts, 208 volts, and 230 volts. Incertain embodiments, each of receptacles in rack power distribution unit170 is on one phase of three-phase input power to the rack powerdistribution unit.

Computer systems 104 may be rack-mountable in rack 102. For example,rails may be installed on the left and right sides of chassis 104 toengage on corresponding rails, slides, or ledges, on left and rightsides of a rack. In certain embodiments, a rail kit may be installed onthe sides of the chassis for the computer systems.

Although only four of computer systems 104 are shown installed in rack102 in FIG. 1 for clarity, a rack system may in various embodiments haveany number of computer systems. For example, rack 102 may hold acomputer system in each 1 U position in rack 102. In one embodiment, arack system has about 20 1 U computer systems.

FIG. 2 illustrates one embodiment of a computer system that can bemounted in a rack. Computer system 104 includes top cover 177. Top cover177 may be pivotally connected to chassis 112 at top cover hinges 179.In FIG. 2, top cover 177 has been flipped away from its closed positionon top cover hinges 179 to expose internal components of computer system104.

Circuit board assembly 114 includes circuit board 180, processors 182,DIMM slots 184, and I/O connectors 186. Circuit board assembly 114 mayinclude various other semiconductor devices, resistors, and other heatproducing components. Circuit board assembly 114, along with othercomponents in chassis 112 (hard disk drives, power supplies) and/orcomponents external to chassis 112, may operate in conjunction with oneanother as a computer system. For example, computer system 100 may be afile server.

In the embodiment shown in FIG. 2, computer system 104 includes onepower supply unit and 12 hard disk drives. A computer system may,however, have any number of hard disk drives, power supply units, orother components. In certain embodiments, a computer system may have oneor more internal fans to promote the flow of air through a computersystem. For example, in certain embodiments, a row of fans may beprovided along the rear edge of computer system 104. In certainembodiments, a computer system may have no fans and/or no disk drives.In certain embodiments, a power supply may be external to a computersystem. For example, in certain embodiments, circuit board assembly 114may receive power from a power supply external to computer system 104(such as a rack-level power supply), and power supply 130 may beomitted.

Heat sinks 189 are mounted on processors 142. Heat sinks 189 maytransfer heat from processors 142 to air inside chassis 112 duringoperation of computer system 100. DIMMs (not shown for clarity) may beinstalled in any or all of DIMM slots 184.

When top cover 177 is in a service position (for example, flipped outover the front panel of computer system 104, such as shown in FIG. 2),top cover 177 may serve as a working tray. The working tray may be usedduring maintenance operations. For example, maintenance personnel mayrest components such as hard disk drives, cables, and circuit boardassemblies on top cover 177. In some embodiments, top cover may includestructural reinforcement, such as ribs, to increase the amount andweight of materials that can be rested on top cover 177. In certainembodiments, top cover 177 includes texturing or non-slip surfaceelements or treatments to help keep working materials such as cables inplace during maintenance.

When computer system 104 is to be reinstalled in rack 102, top cover 177may be flipped back into a closed position on computer system 104. Insome embodiments, top cover 177 includes a latch to secure the top coverin a closed position on computer system 104.

In some embodiments, a rack system is arranged so that air is receivedinto computer systems in a rack on the same side of the rack as theinput/output connections and power connections for the computer systems.In one embodiment, one portion of air may flow into inlets in a chassisfor a computer system and another portion of the air may flow into apower supply unit for the computer system.

In certain embodiments, all of the connections for a server are locatedon a cold aisle for the rack system. Locating all of the connections ona cold aisle may allow maintenance personnel to avoid having to performany operations in a hot aisle. FIG. 3 illustrates one embodiment of adata center having power and data connections on a cold-aisle side of arow of racks. Data center 190 includes row of racks 192, cold aisle 194,and hot aisle 196. Row 192 includes racks 102 and computer systems 104.Each of racks 102 includes rack power distribution units 170 and fans162. Cold aisle 194 may provide a supply of air for cooling computersystems 104 in racks 102. Fans 162 may draw air from cold aisle 194 intoracks 102 and through computer systems 104, and then exhaust air fromthe computer systems into hot aisle 196. In one embodiment, air enteringthe servers from the cold aisle may be at about 35 degrees Celsius. Inone embodiment, air exhausted from the servers into the hot aisle may beat about 50 to 55 degrees Celsius.

FIG. 4 illustrates air flow through a computer system in one embodiment.Power supply unit 116 of computer system 104 includes power supplyenclosure 202, power supply circuit board assembly 204, fan 206, and airvents 208. Power supply circuit board assembly 204 may receive powerfrom input power receptacle 134 and supply power to electricalcomponents in computer system 104. Fan 206 may move air through powersupply enclosure 202 and across heat producing components on powersupply circuit board assembly 204.

In some embodiments, air is drawn into a power supply unit enclosurefrom outside the computer system and expelled into a chassis of thecomputer system. For example, as illustrated in FIG. 4, fan 206 may drawair from outside computer system 104 into power supply enclosure 202through air vents 136, across power supply circuit board assembly, andexpel the air through air vents 208 into the inside of chassis 112 ofcomputer system 104.

In various embodiments, a computer system includes a power supply thatconforms to an industry-recognized standard. In some embodiments, apower supply for a computer system has a form factor in accordance withan industry-recognized standard. In one embodiment, a power supply unithas a standard 1 U form factor. Examples of other standards for a powersupply and/or a power supply form factor include 2 U, 3 U, SFX, ATX,NLX, LPX, or WTX.

In some embodiments, a standard power supply is modified to reverse theairflow for the power supply unit. For example, in the off-the-shelfconfiguration of a standard 1 U power supply with the form factor shownin FIG. 4, fan 206 may pull air into power supply enclosure 202 throughair vents 208 and expel air out of power supply enclosure through airvents 136 (that is, pull air from inside the computer system and expelair to the outside at the front panel of the server). In someembodiments, air flow in the standard power supply may be reversed suchthat fan 206 pulls air into power supply enclosure 202 through air vents136 and out of power supply enclosure 202 through air vents 208 (whichresults in the air flow being from outside the computer system toinside). In one embodiment, reversing the air flow is accomplished byreversing the wiring of the fan from the off-the-shelf configuration. Inanother embodiment, reversing the air flow is accomplished by flippingthe orientation of the fan such that so that air flows in the oppositedirection from that of the off-the-shelf configuration.

In some embodiments, input and output power specifications for a powerrouting device may conform to an industry standard. In one embodiment,the voltages and functions conform to an ATX standard. In various otherembodiments, the output from a power routing device may conform to otherstandards, such as Entry-Level Power Supply Specification, or EPS12V.

In some embodiments, an input power receptacle and an air intake for apower supply are located at a common end of a computer system. Forexample, in the embodiment illustrated in FIG. 4, input power receptacle134 and air vents 136 are located on power supply panel 132. Powersupply unit 116 is oriented in chassis 112 such that power supply panel132 aligns with the input/output panel 120 at front 119 of computersystem 104.

Hard disk drives 118 are mounted on tray 222. Air exiting power supplyenclosure 202 may flow toward the rear of computer system 104 throughchannels 220 under hard disk drives 118. Air passing over circuit boardassembly 114 may also flow toward the rear of computer system 104through channels 220 under hard disk drives 118. Air flowing throughchannels 220 under hard disk drives 118 may remove heat from heatproducing components in hard disk drives 118, such as control componentsand electrical motors. Air may exit from the rear of computer system104. Tray 222 includes rim 221. In some embodiments, rim 221 may blockall or a portion of the air in a chassis from passing over hard diskdrives 118. In certain embodiments, cases of the hard disk drives mayblock all or a portion of the air in the chassis from passing over thehard disk drives. By blocking air from passing over the hard diskdrives, rim 221 may redirect the air to flow through channels 220 underhard disk drives 118.

FIG. 5 illustrates a rear view of computer system 104. Tray 222 for harddisk drives 118 and side panels 223 and bottom panel 224 of chassis 112define openings 225 at rear 226 of computer system 104. Air flowingunder hard disk drives 118 may exit computer system through openings225. Tray 222 may be held under upper tabs 226 of on chassis 112. Tray222 may be secured at the rear of tray 222 by way of screws 227. Screws227 may pass through holes in tray and thread into rear tabs 228 ofchassis 112.

In some embodiments, removing air downstream from of a circuit boardassembly at the bottom of a computer system enclosure may enhance airflow over the circuit board assembly. FIG. 6 illustrates a schematicside view of air flow through a computer system according to oneembodiment. Air may flow over circuit board assembly 114 to the rear ofthe circuit board assembly 114, where the air is drawn under hard diskdrives 118. Removing air downstream passing over a circuit boardassembly near the bottom of a chassis, such through channels 220 underhard disk drives 118, may enhance airflow across circuit board assembly114 by increasing the velocity of the air flow at the surface of circuitboard assembly 114 and/or increasing turbulence of air at or near thecircuit board.

In some embodiments, air leaving exiting a power supply unit may mixwith air used to cool other heat producing components in a computersystem enclosure, such as a central processing unit. FIG. 7 is aschematic top view illustrating air flow through a computer system inone embodiment. One portion of air may enter power supply unit 116through air vents 136 and be expelled into chassis 112 of computersystem 104 through air vents 208. A second portion of air may entercomputer system 104 through air vents 122 and pass over circuit boardassembly 114. The air passing through power supply unit 116 and overcircuit board assembly 114 may mix downstream of the power supply andthe circuit board assembly. Mixing of air from power supply unit 116 andcircuit board assembly may result in more uniform temperatures of airflowing across downstream components, such as hard disk drives 118. Forexample, relatively hot air from power supply unit 116 may mix withrelatively cool air near the rear of circuit board assembly 114. In someembodiments, power cable 230 for power supply unit 116 may serve as adiffuser for air exiting power supply unit 116. In certain embodiments,a computer system may include dedicated elements for mixing air, such asvanes. For example, vanes may be provided on the bottom of chassis 112of computer system 104.

In the embodiments described above, air exhausted from the power supplyunit mixes with air from other components of the computer system. Inother embodiments, however, air from a power supply may be segregatedfrom other air inside a computer system enclosure. For example, in oneembodiment, air exiting from power supply unit 116 may be segregatedfrom other air in chassis 114 and moved to the rear of computer system104 in a separate duct. In certain embodiments, a computer system mayinclude a barrier between the air exiting a power supply unit and otherair inside a computer system enclosure. In some embodiments, a computersystem may include a dedicated fan for cooling of a power supply unit. Adedicated fan may be used to increase air flow over heat producingcomponents in the power supply unit.

In some embodiments, one or more rack power distribution units areprovided at the air intake side of a rack. The rack power distributionunits may be at the same end of the rack as the connections for powersupplies for servers in the rack. Referring again to FIG. 1, system 100includes rack power distribution units 170 on the left and right sidesof rack 102 at the front of rack 102.

In some embodiments, rack power distribution units are attached to arack by way of a coupling device. In some embodiments, a coupling deviceallows for movement or repositioning of the rack power distributionunit, for example, during maintenance operations. FIG. 8 illustrates oneembodiment of a rack power distribution unit on a coupling device thatallows for movement of the rack power distribution unit.

Rack power distribution unit 170 includes rack PDU enclosure 240. Rackpower distribution unit 170 is mounted on bracket 172. Bracket 172 mayserve as coupling device for rack power distribution unit 170. Bracket172 is coupled to rack 102 by way of hinges 242. Rack power distributionunit 170 may be coupled to bracket 172 by engagement of buttons 247 inkeyhole slots 249.

Each of hinges 242 include hinge element 243 and hinge element 244.Hinge elements 244 may be on the rack side of the hinge 242. Hingeelement 243 may be on the bracket side of hinge 242. Hinge elements 243and hinge elements 244 may cooperate to form a hinges 242 betweenbracket 172 and rack 102. Rack power distribution unit 170 may rotate onhinges 242.

In some embodiments, hinge elements 244 may be part of a standard,off-the shelf rack. The locations and dimensions of hinge element 243 ofbracket 172 may be chosen to match the locations and dimensions of hingeelements 243 of the standard, off-the shelf rack. In certainembodiments, a door may be removed from a standard, off-the-shelf rackand replaced with one or more hinge-mounted rack power distributionunits, such as shown above relative to FIG. 8.

In some embodiments, a bracket for mounting a rack power distributionunit may be reversible such that the same bracket can be used on eitherside of rack. For example, bracket 172 may be reversible such thatbracket 172 can be used to couple rack power distribution units 170 onboth sides of rack 102.

In certain embodiments, hinge element 243 may be formed from sheet metalas an integral part of bracket 172. In other embodiments, hinge element243 may be produced as a separate part and then attached to bracket 172,for example, by rivets, screws, or welding.

Rack power distribution unit 170 may include any number of receptacles.For example, with reference to FIG. 1, each of rack power distributionunits 170 on the left and right sides of rack 102 may includes 21receptacles. Rack power distribution unit 170 may receive input powerthrough cable 241. Each of receptacles 176 in rack power distributionunit 170 may be coupled to a different computer system 104 in rack 102by way of one of power cables 178 (in FIG. 8, power cable 178 is shownfor only one such connection for clarity).

FIG. 9 is a top view schematic illustrating one embodiment of a racksystem with rack power distribution units in a normal position. FIG. 10is a top view schematic illustrating one embodiment of a rack systemwith rack power distribution units in a maintenance position. System 100includes rack 102, computer systems 104, and rack power distributionunits 170. Rack power distribution units 170 are coupled to rack 102 byway of brackets 172. When rack power distribution units 170 are in anormal position (as shown, for example, in FIG. 9), rack powerdistribution units 170 and/or brackets 172 may be ininstallation/removal path 248 of computer systems 102. Rack powerdistribution units 170 may be repositioned to a maintenance position (asshown, for example, in FIG. 10). As rack power distribution units 170are rotated with respect to rack 102 on brackets 172, power cables 178may uncoil relative to the rotation axis of the hinge. Uncoiling and/orunwinding of power cables 178 may reduce or eliminate tension in powercable 178 as the receptacles on rack power distribution unit 170 areswung away from computer systems 104.

With rack power distribution units 170 in a maintenance position, any ofcomputer systems 104 in rack 102 may be installed or removed from rack104. For example, computer system 104′ may be slid out of rack 102 whilerack power distribution units 170 are in the maintenance position.

In some embodiments, rack power distribution units are provided on bothends of a rack. The rack power distribution units may be movable withrespect to the rack to facilitate access to servers in the rack.

In some embodiments, movable rack power distribution units are providedon both ends of a cold-cold rack system. FIG. 11 illustrates a schematictop view of one embodiment of a data center having a row of cold-coldrack systems with rack power distribution units on both ends of therack. FIG. 12 illustrates a schematic end view of the data center shownin FIG. 11. Data center 250 includes computer room 251, subfloor chamber252, and plenum 253. Computer room 251 includes cold-cold rack systems254 in common row 255. In one embodiment, cold-cold rack system 254 is ahalf-depth server made by Rackable Systems, Inc. Row 255 separates coldaisles 256.

Each cold-cold rack system 254 includes rack 257 and half depth servers258. Servers 258 may be any height, including 1 U, 2 U, or 3 U. In eachcold-cold rack system 254, mid column 259 is provided or formed in thespace between the front stack of half depth servers 258 and the backstack of half depth servers 258. Servers 258 in cold-cold rack systems254 are cooled by drawing air into rack system 254 on both the front andback of the rack system, and removing the air from mid column 259through the top of racks 257.

To remove heat from servers 258, an air handling system may be operatedto cause air to flow from subfloor 252 to computer room 251 throughaisle floor vents 260. Air from aisle floor vents 260 may pass from coldaisles 256 into cold-cold rack systems 254. In one embodiment, the flowof air in the racks is about 450 cubic feet per minute per rack, perside. Air flows from the front side and back side of rack systems 254through half depth servers 258 to mid column 259. Air in mid column 279passes out through the tops of racks 257.

Cold-cold rack systems 254 include rack power distribution units 170 onfront side 201 and back side 203 of rack 102. Rack power distributionunits 170 are coupled to racks 102 of cold-cold rack systems 254 onbrackets 172. Rack power distribution units 170 and brackets 172 may besimilar to those described above relative to FIG. 8. Brackets 172 mayprovide a hinged connection between rack power distribution units 170and rack 102. Rack power distribution units 170 may be rotated (forexample, by maintenance personnel) to provide access to servers 258.

Air from cold aisles 256 may pass across rack power distribution units170 before passing through servers 258. Cool air circulation in the areaof rack power distribution units 170 may reduce a risk of failure ofpower distribution units 170 (for example, failure due to overheating ofbreakers in the rack power distribution units).

Power cables (not shown in FIGS. 11 and 12 for clarity) may connectinput power receptacles on power supply units 116 in servers 258 topower output receptacles in rack power distribution units 170. FIG. 13is a top view schematic illustrating a power connection for one of racksystems 254 with rack power distribution unit 170 in a closed position.FIG. 14 is a top view schematic illustrating a power connection for oneof rack systems 254 with rack power distribution unit 170 in an openposition. When rack power distribution unit 170 is in an open position,any of the servers in rack system 254 may be removed from rack system254 (as illustrated in FIG. 11).

FIG. 15 illustrates an embodiment of a rack system with two powerdistribution units on a single bracket. Two rack power distributionunits 170 are coupled to bracket 264. Bracket 264 is coupled to rack102. Each of power distribution rack power distribution units 170 mayinclude a series of power receptacles. In one embodiment, each of rackpower distribution units 170 includes 21 receptacles.

In the embodiments shown in FIG. 8-10, bracket 172 couples a rack powerdistribution unit for rotational motion relative to a rack about hinge242. A coupling device may, however, couple a rack power distributionunit for other types of motion relative to a rack. FIG. 16 is aschematic view illustrating an embodiment of a rack system including arack power distribution unit coupled to a rack by way of a linkage. Racksystem 100 includes linkage 266. Linkage 266 couples rack powerdistribution unit 170 to rack 102. Linkage 266 includes links 267 andhinges 268. Rack power distribution unit 170 may be repositioned onlinkage 266 to provide access to computer systems 104.

In certain embodiments, a rack power distribution unit may be coupledfor translation relative to a rack. FIG. 17 is a schematic viewillustrating an embodiment of a rack system including a rack powerdistribution unit in a sliding arrangement. Rack system 100 includesslide coupling 272. Slide coupling includes plate 273 and rail 274. Rackpower distribution unit 170 may couple on rail 274. Rack powerdistribution unit 170 may be slid on rail 274 in the direction of thearrows to provide access to computer systems 104.

FIG. 18 illustrates an embodiment of a rack system that includes frontrack power distribution units and rack doors. Rack power distributionunit 170 is coupled to rack 102 by way of bracket 275. In someembodiments, bracket 275 in connected to rack 102 by way of one or morehinges. Doors 276 may couple to bracket 276. In certain embodiments,bracket 275 may use hinge elements on a standard, off-the-shelf rack. Incertain embodiments, brackets 275 and doors 276 may replace standardrack doors of an off-the-shelf rack.

FIG. 19 illustrates one embodiment of a tray for hard disk drives in acomputer system. Tray 222 includes base 280 and clamping bars 282.Clamping bars 282 may hold hard disk drives 118 (not shown in FIG. 19for clarity) in position on base 280.

Base 280 of tray 222 includes hard disk support plate 281, risers 283,and peripheral frame 277. Risers 283 may space hard disk drives from thebottom of a chassis of a computer system (such as chassis 112). Airpassages 220 may be defined between risers 283 when tray 222 isinstalled in the chassis.

Support plate 281 includes openings 279. Openings 281 may promote thetransfer of heat from heat producing components in hard disk drives toair moving through air passages 220. In certain embodiments, heat sinksmay be provided on the underside of hard disk drives to promote heattransfer to air passages 220. In some embodiments, heat sinks areprovided on tray 222. In other embodiments, heat sinks may be attachedto the hard disk drives in the tray.

Tray 222 includes pads 286 on base 280. Pads 286 may protect hard diskdrives 118 from vibration and/or shock loads. In one embodiment, pads286 are made of a polymeric material.

In some embodiments, the spacing elements and disk drive supportelements of a tray may be integral to one another. In the embodimentillustrated in FIG. 19, for example, support plate 281, risers 283, andperipheral frame 284 may be all formed as one piece. In one embodiment,support plate 281 and risers 283 are integrally formed from sheet metal.In other embodiments, support plate 281 and risers 283 are separateparts, which may be coupled to one another by fasteners, welds, adhesiveor other manner. In various embodiments, the supporting portions and/orspacing portions of a tray may include rails, tubes, rods, bars, or anyother structural elements.

FIG. 20 illustrates one embodiment of a clamping bar in a raisedposition from the base of a tray. Clamping bar 282 may be pivotallycoupled to base 280 by way of pins 284. Clamping bar 282 includes arms287 and cross member 288. Cross member 288 includes channels 289 andwings 290. Channels 289 and wings 290 may combine to form an inverse hatsection on each side of cross member 288. Wings 290 may engage the edgesof hard disk drives to hold the hard disk drives in tray 222.

FIG. 21 illustrates a side view of one embodiment of a tray for harddisk drives. In FIG. 21, locking bars 282 are in a closed position onbase 280 (hard disk drives have been omitted for clarity).

FIG. 22 illustrates a cross section of locking bar on adjacent diskdrives. Each of wings 290 of clamping bar 282 may contact a top surfaceof one of hard disk drives 118. Contact between wings 290 of clampingbar 282 and the top surfaces of the hard disk drives may retain the harddisk drives in tray 222. For some of hard disk drives 118, one edge ofthe hard disk drive is retained by one of tabs 285 on a frame of tray222, and the opposite edge of the hard disk drive is retained by one ofwings 290, as shown in FIG. 4.

In some embodiments, clamping bar 282 may be resiliently biased suchthat wings 290 bear down on the hard disk drives. For example, incertain embodiments, tray 222 may include torsion spring at the pivotalconnection between clamping bar 282 and base 280. The torsion spring mayresiliently bias clamping bars 282 into contact with hard disk drives118.

In another embodiment, torsion springs may resiliently bias clampingbars away from base 280 (for example, to facilitate removal of the harddisk drives).

FIG. 23 illustrates a tray with clamping bars in an open position toallow removal of hard disk drives from the tray. Clamping bars 282 maybe raised to allow installation or removal of hard disk drives 118.

In some embodiments, elements for holding down disk drives can be usedas carrying handles for the hard disk drives, or as carrying handles fora computer system in which the tray is mounted. For example, clampingbars 282 may be grasped by a user to carry hard disk drives 118. In someembodiments, tray 222 is tool-less such that no tools are required toinstall tray 222 or to install or remove hard disk drives from the tray.

FIG. 24 illustrates one embodiment a locking mechanism for hard diskdrive tray. Clamping bar 282 includes pin 291 and pin mounting bracket292. Pin mounting bracket 292 is coupled to clamping bar 282 at the baseof channel 289. Pin 291 is arranged to slide in pin mounting bracket292. Clamping bar 282 includes through hole 293 for pin 291.

When clamping bar 282 is lowered to a closed position on hard diskdrives 118, pin 291 and through hole 293 may align with hole 294 in base280 of tray 222. Pin 291 may be advanced into hole 294. In certainembodiments, pin 291 may be spring-loaded into engagement in hole 294 ofbase 280. Engagement of pin 291 in hole of base 280 may retain clampingbar 282 in a locked position on hard disk drives 118.

Although in FIG. 24, clamping bar 282 is locked with pin 291, a trayassembly for hard disk drives may, in various embodiments, include otherlocking mechanisms and locking elements. Examples of locking mechanismsand elements include screws, cams, wedges, springs (for example, leafsprings), and detent mechanisms. In certain embodiments, hard diskdrives may be coupled in a tray assembly by way of an interference fitbetween the hard disk drives and the tray.

Although the embodiments described above, the tray was used to mounthard disk drives in a raised position, in various embodiments a computersystem may include any of various data storage devices mounted in araised position.

Although in the embodiments, described above, all of the hard diskdrives were mounted on a tray, in various embodiments, hard disk drivesor other data storage devices may be mounted to a chassis using othermounting elements. For example, hard disk drives may be mounted onsquare tubes that support the drives and raise the drives above thebottom of a chassis.

In some embodiments, a tray may provide structural reinforcement forcomponents in a chassis, such as hard disk drives. Referring again toFIG. 5, risers 283A, 283B, and 283C may couple with bottom panel 296 ofchassis 112. In some embodiments, risers 283A, 283B, and 283C may reston bottom panel 224. In other embodiments, risers 283A, 283B, and 283Cmay be attached to bottom panel 224, such as by screws, welding, orother manner of attachment. Side panels 223 and/or upper tabs 226 ofchassis 112 may couple with peripheral frame 277 of tray 222. Elementsof tray 222 and chassis 112 may combine to form a box section mountingfor hard disk drives 118. For example, bottom panel 296, risers 283A and283C, and support plate 281 may combine to form a rectangular boxsection. A box section may reduce deformation of a chassis, such assagging of bottom panel 224, which might occur if hard disk drives 118were installed directly on bottom panel 224 of chassis 112.

In some embodiments, a rack system includes rack-mounted fans externalto computer systems in the rack. The rack-mounted fans may provide airflow through the computer systems. In the embodiment illustrated in FIG.1, for example, system 100 includes fans 162.

FIG. 25 illustrates a rear view of one embodiment of a rack system.System 100 includes rack 102 and rear door 160. Rear door 160 coupleswith rack 102 on hinges 148. Fan modules 300 couple with, and aresupported by, rear door 160.

FIG. 26 illustrates one embodiment of a fan module for a rack. Fanmodule 300 includes one or more fans 302 and case 304. Case 304 includeshandle 305.

Fan module 300 may be mountable in a door of a panel (see, for example,rear door 160 shown in FIG. 25 and FIG. 27). Fans 302 may provide airflow in rack 102.

In some embodiments, fans 302 are alternating current (AC) fans. In oneembodiment, fans 302 have an input voltage rating of about 100V-120 V.In one embodiment, fans 302 have an input voltage rating of about 230 V.Fans 302 may receive power from rack level power distribution units(such as rack power distribution unit 170 described above relative toFIG. 1). In some embodiments, fan modules 300 in a rack are hotswappable. In some embodiments, a manual power switch is provided foreach of fan modules 300.

In one embodiment, each of fans 302 operates at a flow rate between 50to 100 cubic feet per minute. In one embodiment, each of fans 302operates at a flow rate about 200 cubic feet per minute.

FIG. 27 illustrates one embodiment of a door for supporting fans in arack system. Door 160 includes fan module socket 320. Fan module socket320 includes guides 322, openings 324, and blind-mate connectors 330.Each of guides 322 may engage a complementary channel on one of fanmodules 300 when the fan module is installed in door 160. Each ofblind-mate connectors 330 may couple with a complementary connector onone of fan modules 300 when the fan module is installed in door 160. Insome embodiments, blind-mate connectors 330 may receive power through acable harness (not shown in FIG. 27 for clarity). In some embodiments,the cable harness may supply each of receptacles 330 with power from anoutput power receptacle in a rack power distribution unit (such as rackpower distribution unit 170).

Each of fans 302 may provide air flow for more than one computer systemin rack 102. For example, as illustrated in FIG. 1, rack 104 may includecomputer systems 102, which may each be 1 U in height. In someembodiments, fans 300 combine to create a low pressure region at therear of the rack.

In the embodiment shown in FIG. 1 and FIG. 25, a system includes threerows spaced from the top of rack to the bottom, with each row includingtwo fans. In various embodiments, however, a system may have any numberof fans. In some embodiments, a system has three or more fans per row.In some embodiments, a system has only one fan per row.

In some embodiments, rack-mounted fans for a system may be N+1redundant. For example, in the embodiment shown in FIG. 1 and FIG. 25,if one of the six fans fails, the remaining five fans may provideadequate cooling for the system.

In some embodiments, fans are mounted in a rack at an angle relative tovertical. For example, as illustrated in FIG. 1, fans 162 are mounted atan angle relative to vertical. In certain embodiments, the angles of thefans in a rack are chosen to create a relatively uniform low pressureregion at the rear of a stack of computer systems. In one embodiment,fans are angled at about 45 degrees from vertical. In anotherembodiment, fans are mounted at an angle that is near horizontal (forexample, more than 80 degrees from vertical). The angles of fans in arack may be the same or different.

FIG. 28 illustrates a rear view of fan module 300. Rear panel 332includes connector 334. Connector 334 may couple with a complementaryconnector on of a mounting panel, such as connector 330 on rack door160. Channels 335 may be provided on both sides of fan module 300.Channels 335 may engage on guides 322 in fan module sockets 320 of rackdoor 160 when fan module 300 is installed in rack door 160.

FIG. 29 illustrates a side view of a fan module including a mountingchannel. To install fan module 300, mounting channels 335 may be engagedon a pair of guides 322 shown in FIG. 27. Captive screws 337 may threadinto tapped holes 336 of fan module sockets 320. In other embodiments,various other mechanisms may be used to couple a fan module to a rack.Such mechanisms may include, for example, a latch, a cam, or a clip. Inone embodiment, a system includes a depressible latch mechanism forcoupling a fan module to a rack. The latch mechanism may be operablewith one hand to latch or remove the fan module from the rack.

In some embodiments, the angle of a rack-mounted fan is adjustable. FIG.30 illustrates one embodiment of a fan with an adjustable mounting.System 338 includes fan module 340. Fan module 340 is rotatably coupledon fan module mount 342. Fan module mount 342 is mounted in rear panel344 of rack 345. Fan module 340 includes fan 346.

Fan module 340 may be rotatably coupled to fan module mount 344. Fanmodule 340 is coupled to fan module mount 342 at pivot connection 348.The angle of fan module 342 may be adjusted by rotating fan module 340on pivot connection 348.

Rack 345 includes roof fan module 360 mounted in roof 364. Fan 362 ofroof fan module 360 may operate in combination with one or more fans 340in rear panel 344 to provide a low pressure region in the rear portionof rack. In one embodiment, a rack includes one row of fans on the roofof a rack and two rows of fans spaced from top to bottom in the reardoor of the rack.

FIG. 31 illustrates an adjustable fan adjusted to a vertical angle. Inone embodiment, locking mechanism 350 is released to allow the angle offan module 340 to be adjusted. Fan module 340 may be rotated on pivotconnection 348 to a vertical angle. Once fan module 340 is in position,locking mechanism 350 may be operated to lock fan module 340 at thedesired angle.

In some embodiments, a system may include variable speed fans. Incertain embodiments, power switching and/or fan speed may be controlledautomatically. Fans may be controlled individually, or in groups of twoor more fans. In some embodiments, fans are controlled based on sensorsdata (for example, temperature sensors in the rack).

In some embodiments, one or more fans for a rack system may becontrolled via a control system. Referring to FIG. 3, for example, rack192A includes control system 380. Control system 380 may be coupled tofans 162. In certain embodiments, a control system includes at least oneprogrammable logic controller. The PLC may receive measurements ofconditions in the rack or at other locations in a data center. A PLC mayreceive data corresponding to air flow rate, temperature, pressure,humidity, or various other operating or environmental conditions.

In one embodiment, the PLC receives data from one or air flow sensorsthat measure airflow in the rack. Based on sensor data, the PLC maycontrol parameters such as fan speed, as appropriate for the prevailingoperational conditions. In another embodiment, the PLC receives datafrom one or more temperature sensors that measure temperature in therack and/or at other locations in a data center. In certain embodiments,a PLC may modulate dampers between open and closed positions to modulateairflow, as appropriate for the prevailing operational conditions.

In some embodiments, a PLC may receive data from thermal sensors in arack power distribution unit. In certain embodiments, a PLC may controlswitching in a rack power distribution unit.

In certain embodiments, angular adjustment of a fan may be automated.For example, the angle of fan module 340 relative to rack 102 may becontrolled with an actuator coupled to the fan module. The actuator maybe controlled by a PLC to adjust the angle of fan module 340.

In some embodiments, a system may include fan failure detection devices.In one embodiment, each of fan modules 300 is provided with a Halleffect sensor. The Hall effect sensor may provide a signal to a controlsystem that a fan is not operating.

In some embodiments, one or more fans of a rack system may be providedwith a device that automatically shuts off air flow through the fan ifthe rate of air flow through the fan drops below a predeterminedthreshold. For example, referring to FIG. 25, fan module 300′ mayinclude louvers 370. Louvers 370 may automatically close if air flowthrough fan module 300′ drops below a predetermined level. For example,if fan module 300′ fails, louvers 370 may automatically close to shutoff air flow through the fan. Automatically shutting off air flow to arack-mounted fan may reduce back flow of air into rack 102 in the eventof a failure of fan module 300′. In some embodiments, all of the fanmodules on a rack may include louvers for shutting off air flow in theevent of a fan failure.

In certain embodiments, air flow through computer systems in a rack maybe provided using fans internal to the computer systems instead of, orin addition to, rack-mounted fans. For example, a series of fans may beprovided at the rear of the chassis of each of the computer systems in arack (for example, downstream from hard disk drives 118 shown in FIG.1). In certain embodiments, air flow for computer systems in a rack maybe produced by an air handling system external to the rack.

In some embodiments, a power system for fans in a rack may includeback-up power. Back-up power may be implemented, for example, in theevent of a failure of a primary rack power distribution unit. In someembodiments, power to fans in a rack is automatically switched from aprimary rack power distribution unit to a reserve power rack powerdistribution unit upon failure of the primary rack power distributionunit.

Referring again to FIG. 3, data center 190 includes fan powerdistribution system 378. Fan power distribution system 378 includesautomatic transfer switch 390, primary power cable 392, reserve powercable 394, and fan power cable 396. During normal operation, fans 162may receive power from primary rack power distribution unit 170A on theleft side of rack 192. Power from primary rack power distribution unit170A may be supplied through primary power cable 392, automatic transferswitch 390, and fan power cable 396. In the event of a failure of rackpower distribution unit 170A (or in the primary power system upstreamfrom primary rack power distribution unit 170A), automatic transferswitch 390 may automatically switch to reserve power. In the reservepower mode, reserve rack power distribution unit 170B may supply powerto fans 162 through reserve power cable 394, automatic transfer switch390, and fan power cable 396. In some embodiments, each of primary rackpower distribution unit 170A and reserve rack power distribution unit170B are movably coupled to the rack, such as rack power distributionunit 170 described above relative to FIG. 1.

Automatic transfer switch 390 may be placed or mounted in any of varioussuitable locations. In one embodiment, automatic transfer switch 390 ismounted on the roof of a rack. In another embodiment, automatic transferswitch 390 is mounted on the rear door of a rack.

FIG. 32 illustrates a method of cooling components in a rack-mountedcomputer system according to one embodiment. At 400, air moving devicesare provided for heat producing components in a rack-mounted computersystem. The air moving devices may be, for example, a fan in a powersupply unit and/or a fan mounted in the rack. In some embodiments, anair moving device is an internal fan in a power supply unit that hasbeen modified to reverse the direction of air flow in the power supplyunit.

At 402, air is drawn from outside the rack-mounted computer system intoa power supply enclosure for the power supply unit. At 404, air isexpelled from the power supply enclosure into an enclosure for therack-mounted computer system.

In an embodiment, rack power distribution units in a rack are moved toallow access to computer systems in the rack so that reconfiguration ormaintenance operation can be performed on the computer systems. FIG. 33illustrates one embodiment of a reconfiguration or maintenance operationthat includes moving rack power distribution units to access computersystems in a rack. At 410, rack power distribution units installed inthe rack are moved from an operating position to a maintenance positionwhile the rack power distribution units remain coupled to the rack. Forexample, rack power distribution units may be rotated on brackets thatare coupled to the rack by way of hinges. Moving the rack powerdistribution units may take the rack power distribution units out of aninstallation/removal path for computer systems in the rack.

At 412, a computer system in the rack is at least partially installed orremoved while the rack power distribution unit is in the maintenanceposition. In some embodiments, a computer system is entirely removedfrom the rack and replaced by another computer system. In someembodiments, a reconfiguration or maintenance operation is performed ona computer system is performed while the computer system is partiallyinstalled in the rack (for example, slid out but still on the rails).For example, a cover on the computer system may be removed and a circuitboard assembly, power supply unit, or a hard disk drive, may bereplaced. In either case, once the computer systems have been placed orreplaced in their fully installed positions, the rack power distributionunits may be returned to their operating positions at 414.

In an embodiment, a method of cooling computer systems in a rackincludes directing air under hard disk drives in a rack-mountablecomputer system. FIG. 34 illustrates a method of cooling hard diskdrives by flowing air under the drives according to one embodiment. At420, air is drawn into a chassis for a rack-mounted chassis computersystem. At 422, air passing into the chassis may flow over circuit boardassemblies and/or through an enclosure for a power supply unit. At 424,air is directed into passages under hard disk drives of the computersystem. In some embodiments, air is directed downwardly into thepassages. In some embodiments, some or all of the air passing through achassis may be blocked from passing over the hard disk drives. Blockingair from passing may force more air to flow under the hard disk drivesand/or increase the velocity of air flowing under the hard disk drives.

At 426, heat from heat producing components on the hard disk drives maytransfer to air in the passages under the hard disk drives. At 428, airis removed from the passages.

In some embodiments, directing the air under the hard disk driveincludes pulling air downwardly into passages under hard disk drives.For example, in the embodiment shown in FIG. 6, air may be drawndownwardly within chassis 104. In some embodiments, air is pulledthrough with rack-mounted fans at the rear of a rack, such as fans 162shown in FIG. 1.

In an embodiment, a method of cooling rack-mounted computer systemsincludes using rack-mounted AC fans that move air through multiplecomputer systems mounted in the rack. FIG. 35 illustrates a method ofcooling computer systems using rack-mounted fans according to oneembodiment. At 430, AC fans are coupled to a rack. In some embodiments,the fans are coupled to the rack in an angular orientation relative tothe rack.

At 432, the AC fans are operated to move air through computer systems inthe rack such that at least one of the AC fans moves air throughmultiple computer systems mounted in the rack. For example, a single fanmay move air through rack-mounted computer systems at two or morepositions in the rack (for example, two or more servers located one ontop of another in the rack).

At 434, a failure is detected in one of the fans. At 436, a hot-swap isperformed for the failed fan. During the hot-swap, the fan module thathouses the failed fan is removed and replaced. During such removal andreplacement, power may be maintained at the location of the fan moduleand/or to other fans in the rack.

In various embodiments described above, rack power distribution unitsare installed on brackets near the ends of rack systems. A rack powerdistribution unit may, however, be installed in any part of a rack invarious embodiments. For example, a rack power distribution unit may beinstalled on a hinged bracket near the middle of a rack.

Although in the embodiments described above, some of the computersystems have been described as 1 U in height, computer systems may invarious embodiments be 2 U, 3 U, or any other height or dimensions.

Although the embodiments above have been described in considerabledetail, numerous variations and modifications will become apparent tothose skilled in the art once the above disclosure is fully appreciated.It is intended that the following claims be interpreted to embrace allsuch variations and modifications.

What is claimed is:
 1. A system comprising: a rack; computer systemsmounted in the rack from a front side of the rack and from a back sideof the rack on one or more same levels of the rack; a rack powerdistribution unit coupled to the rack on the front side of the rack,wherein the rack power distribution unit is configured to supply powerto a portion of the computer systems mounted in the rack; and anotherrack power distribution unit coupled to the rack on the back side of therack, wherein the other rack power distribution unit is configured tosupply power to another portion of the computer systems mounted in therack.
 2. The system of claim 1, wherein the system is configured suchthat: air is received into the rack from a cold aisle on the front sideof the rack and air is received into the rack from a cold aisle on theback side of the rack.
 3. The system of claim 2, wherein the computersystems mounted in the rack are half-depth servers, wherein the portionof the computer systems that is supplied power from the rack powerdistribution system on the front side of the rack comprise half-depthservers that receive air from the cold aisle on the front side of therack; and wherein the portion of the computer systems that is suppliedpower from the other rack power distribution system on the back side ofthe rack comprise half-depth servers that receive air from the coldaisle on the back side of the rack.
 4. The system of claim 1, whereinone of the computer systems mounted in the rack is a half-depth serverthat comprises: a tray comprising risers configured to elevate the trayabove a surface of a chassis of the half-depth server; and mass storagedevices coupled to the tray at an elevated position above the chassis ofthe half-depth server, wherein the tray is configured to permit air toflow in a space between the tray and the chassis of the half-depthserver to remove waste heat from the mass storage devices coupled to thetray.
 5. The system of claim 4, wherein the half-depth server furthercomprises: a circuit board assembly communicatively coupled with themass storage devices; and a power supply unit; wherein the half-depthserver is configured to receive air from a cold aisle and direct aportion of the air across the circuit board assembly and a portion ofthe air across the power supply unit, wherein the half-depth server isfurther configured to direct the air that has passed across the circuitboard assembly and the air that has passed across the power supply unitinto the space between the tray and the chassis and across the massstorage devices to remove waste heat from the mass storage devices. 6.The system of claim 1, wherein the rack further comprises: a mid column,wherein the mid column is configured to receive heated air that haspassed across mass storage devices of half-depth servers mounted in therack and direct the heated air toward a top side of the rack.
 7. Asystem comprising: a rack configured to mount computer systems in therack from a front side of the rack and from a back side of the rack onone or more same levels of the rack, wherein the rack comprises: acoupling mechanism configured to couple a power distribution unit to therack on the front side of the rack to provide power to at least aportion of the computer systems; and an additional coupling mechanismconfigured to couple an additional power distribution unit to the rackon the back side of the rack to provide power to at least anotherportion of the computer systems.
 8. The system of claim 7, wherein therack comprises: a mid column, wherein the mid column is configured toreceive heated air that has passed across mass storage devices ofhalf-depth servers mounted in the rack and direct the heated air towarda top side of the rack.
 9. The system of claim 8, further comprisinghalf-depth servers mounted in the rack, wherein one of the half-depthservers comprises: a tray comprising risers configured to elevate thetray above a surface of a chassis of the half-depth server; and massstorage devices coupled to the tray at an elevated position above thechassis of the half-depth server, wherein the tray is configured toreceive air from a front side of the rack and direct the air to flow ina space between the tray and the chassis of the half-depth server toremove waste heat from the mass storage devices coupled to the tray; andwherein another one of the half-depth servers comprises: a traycomprising risers configured to elevate the tray above a surface of achassis of the other half-depth server; and mass storage devices coupledto the tray at an elevated position above the chassis of the otherhalf-depth server, wherein the tray is configured to receive air from aback side of the rack and direct the air to flow in a space between thetray and the chassis of the other half-depth server to remove waste heatfrom the mass storage devices coupled to the tray.
 10. The system ofclaim 9, wherein the half-depth server or the other half-depth serverfurther comprises: an opening in the tray configured to permit a portionof the air flowing in the space between the tray and the chassis to flowthrough the opening and across a mass storage device coupled to thetray.
 11. The system of claim 9, wherein the half-depth server or theother half-depth server further comprises: clamping bars pivotallycoupled to a base of the tray and configured to secure the mass storagedevices coupled to the tray between the base of the tray and theclamping bars.
 12. The system of claim 7, wherein the rack is configuredto mount: a set of the half-depth servers in the rack from the frontside of the rack; and another set of the half-depth servers in the rackfrom the back side of the rack.
 13. A half-depth server comprising: achassis having a depth equivalent to half a depth of a standard rack; acircuit board assembly mounted in the chassis; a tray comprising risersconfigured to elevate the tray above a surface of the chassis; and massstorage devices coupled to the tray.
 14. The half-depth server of claim13, wherein the tray further comprises: openings configured to permit aportion of an air flow in a space between the tray and the chassis toflow through the openings and across respective ones of the mass storagedevices coupled to the tray.
 15. The half-depth server of claim 13,wherein the system further comprises: pads coupled between the massstorage devices and the tray, wherein the pads are configured todissipate vibrations.
 16. The half-depth server of claim 13, wherein thesystem further comprises: clamping bars pivotally coupled to a base ofthe tray.
 17. The system of claim 16, wherein the clamping bars furthercomprise: multiple wings configured to secure the mass storage devices.18. The system of claim 17, wherein one of the wings further comprises:a locking bar having an inverse hat shape, wherein the locking bar isconfigured to engage edges of the mass storage devices to secure themass storage devices.
 19. The system of claim 16, wherein the clampingbars further comprise: a locking mechanism configured to secure theclamping bar to the base of the tray in a closed position.
 20. Thesystem of claim 16 further comprising a biasing mechanism configured tobias the clamping bar against the mass storage devices to secure themass storage devices.